BBS in a Box 15/BBS in a box XV-2.iso/Files II/Newton/Resources/Creative Digital Systems/CreativeDig.sit/RTFMDesc.txt

Text File | 1994-11-22 | 17.0 KB | 331 lines | [TEXT/R*ch]

NewtRTFM: Case Study of a Runaway Newton Application
Sandeep Shah
K2 Consultants, Inc.
[e-mail]

THIS ARTICLE WAS FIRST PUBLISHED IN PDA DEVELOPERS (FORMERLY PIE DEVELOPERS)
MAGAZINE. COPYRIGHT (C) 1994 BY CREATIVE DIGITAL SYSTEMS. ALL RIGHTS RESERVED.
CONTACT CREATIVE DIGITAL SYSTEMS AT [e-mail] FOR MORE INFORMATION.

NewtRTFM is a Newton-resident portable reference for the Newton developers.
You may ask: why do I care? (Translation: why isn't it in the ad pages that I
can conveniently skip?) A fair question. A discussion about why I decided to
develop it and how I went about doing it can induce important lessons about
the stages of the design and implementation of a large PDA application.
Hopefully, my replay of the (loud) thinking process will help you see a few
parallels in the nutty problems you are trying to crack yourself.

HOW IT ALL STARTED
Since August 1993, Newton pioneers have had an interesting love-hate
relationship with Newton Programming. I am talking about the Newton Toolkit
(NTK). It was clear that the Newton had potential, and it was also clear that
the development environment would have to mature, especially the alpha
documentation that accompanied the beta software.
    The following problems hindered the learning process in the first few
months:

* missing or misleading information about functions, protos, classes,
behavior and other NewtonScript constructs;
* inconsistencies in function names;
* confusion about choosing methods instead of global functions;
* some not-so-obvious bugs in the early incarnations of the ROMs and
Platform files;
* no effective mechanism to print and study program listings.

In spite of the furious efforts by PIE DTS to answer questions through Q&As
and the net, the basic task of pulling it all together remained a daunting
one.
    This is where NewtRTFM enters. I wanted to have a single source
documentation that would be as portable as the Newton itself. That way I
could convert a little bit of spare time at the oddest of places into a
productive time to work on my favorite Newton project.
    Every developer knows how precious desktop screen real estate is when you
are in the middle of development. An electronic reference on the host would
be a bit inconvenient, disorienting and intrusive. Putting the reference on
the Newton, however, seemed like a good solution - when you are busy using
the Mac keyboard and wondering about the most appropriate function to use,
your Newton is just sitting there. Even when you're not near your development
environment, a Newton-based reference can still be useful.

BASIC DESIGN DECISIONS
Being from the Unix world, I have a strong attachment to the man pages that
provide on-line documentation about commands, system calls, arguments, return
values and other essential topics. I decided that this approach, coupled with
the combining of related topics into various categories, could make the whole
application tremendously useful for developers from beginning to expert
level.
    The next important decision was deciding what information to include. In
order to be useful to a programmer NewtRTFM had to include all functions,
methods, protos, view classes, behavior slots and so on. This had a direct
bearing on the navigational structure. While it's easier to enter a name on a
keyboard-based machine, typing a name on a Newton is hardly an effective way
to look up a topic. (Letting a user say "show SendRemoteControlCode" through
the note pad can be a cute demo, a la "fax Bob," but it's not really
convenient). Besides, we need to consider the cases where a topic name is
known, partially known or totally unknown.
    Taking a cue from indices (as opposed to tables of content), I decided it
was easiest to split the name space by the first letter of each topic name.
Another way to organize that information would be to filter it by various
categories, such as "System Messages." The important lesson here is to
provide fast paths without relying too much on the information that needs to
be written or typed in.
    Next problem was the content within these topics. I decided that the
content should be minimal - a lot of verbiage would mean a lot of storage
which would render the project practically impossible. Also, it would make it
much harder to locate needed information quickly. (A serious modern-day
problem is not one of no information, but of too much information).
     I decided to compartmentalize the information for every topic so you
could quickly zero in on the needed information. For example, you might know
that the Array() function creates an array, but you need to know the type and
the order of each argument. On the other hand you may need to brush up on
FindStringInFrame() by reading a brief description. The other two useful
items I decided to include were a sensible example and personal annotation.

An Application, not a Book
Let me take a quick detour to list my reasons for not doing a Newton
book.
    At  first glance, this may seem like an ideal situation to create a
Newton Book. However, after having worked with various hypertext-based
programming references as a user, I decided against it for a few reasons:

* Linked topics and navigation have a disorienting effect on the reader
because of the lack of a relative context and locality.
* When the task is for reference, you are interested in zeroing on the
needed information quickly.
* The size of the book would be very large. As you see later, some of the
tricks I used to save space may not lend themselves well to the Bookmaker
format.

PUTTING IT ALL TOGETHER
Enough rambling about the preparation. Let's delve into the interface I came
up with (see Figure 1). Note that after going through multiple iterative
experiments, I decided that the most effective interface was to have a
wallpaper-style interface (much like the built-in paper Roll), allowing
enough hooks to let you do your routine tasks.
    With this interface, the main portion of the screen shows the details for
a particular topic. There are four tabs on the right-hand side of the main
view - info, args, show and note - for viewing a description, an argument
summary, an example and your personal notations. There are also four primary
navigation aids:

* the alphabetic palette at the bottom lets you selecting a topic
alphabetically;
* the All button lets you select a subset of topics by category;
* the GoTo button lets you look up a topic by partial string match; and
* the arrows at the top of the screen let you sequence through the current
set of topics.

These navigation aids satisfied my need to find any topic or detail quickly.
    I first implemented a small floating alphabet that showed "A" through "Z"
like the Name application. If you tapped on a letter it displayed a popup of
all the topics starting with that letter. Selecting an item switched to that
topic. This approach had two problems:

* A child view can't be bigger than its parent view - I had to change the
alphabet's viewBounds with SetValue() just before showing this view. This was
slow and distracting.
* I realized that any reference session consists of looking up multiple
topics - the topic information stays on the screen all the time. The alphabet
palette end up being an annoying distraction.

MAJOR CHALLENGES
Now lets look at the serious challenges and what I tried to counter them.

Soup or Not?
NewtRTFM's data is primarily read-only. Looking at soup overhead and
performance, I decided to put the data within the package itself. I
experimented with some homegrown indexing that is easy to implement in
NewtonScript because of the flexibility of frames and arrays. (Most of my
experiments are similar in spirit, although not as elegantly executed, to
those outlined in "Lost In Space" by Mike Engber in PIE Developers 2.3). I
chose a hybrid method using a "frame-of-array-of-frames" with "meta-indexing"
that is hard coded to my needs.

A Special Popup
One of the important navigational features is the alphabetic palette that
lets you see a popup of all topics that begin with the letter you tap. A
popup is simple to do with the DoPopup() function. However, it does not
handle the case where the items span multiple screens. I had to create a
multi-screen popup of my own (see Figure 2).
    I used a view based on clPickView with a method :DoPopup that takes the
same arguments as DoPopup() and add the meta items [--Next--] and [--Prev--] in
the subset of the large list. With a little logic in the pickActionScript to
detect if one of these meta items is tapped.
    (Due to many requests, I am in the process of finalizing this popup as a
public domain user proto that can be used by anyone who is interested).

Runtime vs. Compile-time Trade-offs
After a brief flirtation with building the indexing structures in the
InstallScript, I stumbled upon the wonderful AfterScript. This is where most
of my raw data massaging occurs. Whatever you do in an InstallScript has
runtime implications (both time spent while installing and resetting the
Newton, as well as the RAM used). It's better to do those things that are
known at build time using an AfterScript. Note that most of the data
management and utility functions and the NewtonScript language constructs are
available in the NTK environment and can be leveraged in an AfterScript.
Also, it's possible to provide cross references to objects that reside on
your built package. (The next section of this article shows another possible
use of AfterScripts).
    As an example, if you have some data under development, and you want to
select only certain data on the basis of some compile-time constant, you can
use an AfterScript in the template that contains the selection information. I
use this technique to compile the project for a demo version of NewtRTFM (as
well as for a quick compile) by filtering out most of the data while building
the project.

// The following file contains data in a large
// frame with one slot for each letter of the
// alphabet. For each alphabet, there is an array
// of frames, one frame for each entry
//
// Load command will have this large frame in
// allData
//
Load (HOME & "allData.f");

//
// This is the AfterScript for the base view
// Note that this is not a func, just a compound
// statement
//
AfterScript:
    begin
        // Iterate over all "letters" to filter data
     foreach alphaSym in
        ['a, 'b, 'c, 'd, 'e, 'f, 'g, 'h, 'i, 'k,
        'l, 'm, 'n, 'o, 'p, 'q, 'r, 's, 't, 'u,
        'v, 'w] do
     begin
        if (allData.(alphaSym)) then
        begin
            // "print" is useful for printing the
            // compile time stuff to the inspector!
            print("Slot" && alphaSym && "present")
             abc := []; // none to start with
             foreach item in allDope.(alphaSym) do
             begin
                 // now decide if we want this or not
                // 'kHasStuff is another script in the
                // Project Data that returns TRUE if
                // this item is to be included, NIL
                // otherwise
                 if (Call kHasStuff with (alphaSym, item)) then
                 begin
                     AddArraySlot(abc, item);
                 end;
             end; // foreach item
             // replace the slot
             allDope.(alphaSym) := abc;
        end;    // if (allData.(alphaSym))
    end;    // foreach all symbols
end;    // End of AfterScript

Size
As I started stuffing NewtRTFM with its contents, the package started to
bloat, thanks to Unicode's two-byte characters. This is where the duo
StuffCString() and ExtractCString() came to my help. Since there is no
support for doing this directly through NTK (unless I surround all my string
with StuffCString), I resort to compressing the data using StuffCString in
the same AfterScript mentioned earlier. This almost halves the size of the
program, saving me almost 600K bytes in the final package.
    Since C strings are reasonable candidates for compression, the final
package gets compressed to nearly half the size of the uncompressed data. In
my experiments, by keeping the package uncompressed, the speed improvement
came to just about 10%, hardly noticeable during the interaction. Keeping it
compressed clearly has much more merit.

Live Examples
Apple provides many samples illustrating various NewtonScript features. I
found it hard to try and refer to something effectively while developing an
application. For example I may want to check out some sample which includes
clParagraphView. Usually it's cumbersome in the middle of development to go
find the correct sample and extract the right feature from it. I wanted to
provide some sort of link to useful samples.
    Since it is not practical to keep hundreds of full-fledged samples on a
Newton, I required an alternative. Once again keeping with the theme of
supplying just the needed information, I included an example that would
illustrate the exact usage for a topic. Most of the topics that have a visual
or sound effect are also live, meaning they execute automatically when you go
to the show page.
    For example if you are looking at building a view using
protoExpandoShell, wouldn't it be nice to see a sample that shows how to
create one? That's exactly what I wanted NewtRTFM to do. Go to the topic,
open the show page and you see an example, complete with protoTextExpando,
protoPhoneExpando and protoPhoneExpando children (see Figure 3). If you want
to see behind the curtains, just close the view and you find the NewtonScript
source code that shows you how it was done (see Figure 4). If you want to use
the text, there is a simple way to cut and paste it to the Inspector or to
ViewFrame.

OTHER TASKS
Let's look at some of the other things you may need to do while developing an
application:

* open the standard Newton programs such as Names, Dates;
* open and close the Newton Toolkit slip,
* open other applications that are in the Extras drawer;
* open the application you're working on; or
* write notes.

The first task is easily handled, thanks to the Newton's fixed button bar. I
decided to include some shortcuts to make it easy to accomplish some of these
other tasks:
    The three "holes" on the left side of NewtRTFM's page are ideal
candidates for buttons that I can attach a function to:

* The top button launches the Toolkit. It's marked with "i" for Inspector.
* The bottom button ("a" for Applications) displays an alphabetical popup
list of all the programs in the Extras drawer. You can start any of those
applications from the popup.
* The remaining button ("e" for Extras and Express) provides an
interesting feature. Until you use the Extras button to launch an
application, this button brings up the Extras drawer. After you open an
application using the Applications button, this button acts as an Express
launcher for that program. Usually this is the program under development.

INTERESTING BUGS
Well, are bugs ever really interesting? (Yes, but only in retrospect).
NewtonScript is deceptively simple. But bugs in large applications are hardly
that. Here is a quick list of what I faced and the reasons.

ClEditView
This one drove me nuts. Normally you wouldn't have a viewClickScript defined
for your view based on this class, because the default action is good enough.
But watch out if you have viewClickScripts in the parent hierarchy. If any of
them return TRUE, no matter how "distant" the grand parent is, the default
action for the clEditView is dropped. The end result is no ink or data in the
view. Although now it seems trivial and obvious, it wasn't, until I tried
everything else under the sun.

Sort()
I encountered another bug as a result of ignoring an important side effect of
the Sort() function. I used a simple-looking sort on the extras array in the
global heap:

    local apps := sort(extras, '|<str|, 'text);

so that I could iterate over each item to collect names to be displayed in a
popup. Can you see the problem here right away? If you don't, write "Sort is
destructive to the source array" 1000 times. The interesting thing was that
it wouldn't cause any problems until I tried to delete an application. Then
my Newton would freeze and the program would vanish, but it's memory would
never be released.

My Newton has Amnesia
I had a central script that I would call for every display event in a topic.
I found that every time I used it, there would be a leakage of frames heap
memory by up to 1K. I wrote a second application that used the data from
NewtRTFM and displayed it in a similar view hierarchy in a round-robin
fashion. No losses. I rewrote the script by splitting it into multiple pieces
(same code) and the leakage went away. The only reason I can suggest for the
leak was a large number of locals (around 21) that I was using in the
original script. There's no definite conclusion I can draw here, but I
learned to keep things small and simple.

CONCLUSION
This turned out to be a very interesting project, enriching my knowledge
about Newton development from all possible angles. Trying to come up with
samples that make sense and work interactively proved to be a challenging
task. No matter what, the Newton is a fun platform to work with and I hope
that NewtRTFM helps convey that image.